MATLAB Deep Learning Toolbox™ User's Guide [R2022b ed.]

Deep Networks
	Deep Learning in MATLAB
		What Is Deep Learning?
		Start Deep Learning Faster Using Transfer Learning
		Deep Learning Workflows
		Deep Learning Apps
		Train Classifiers Using Features Extracted from Pretrained Networks
		Deep Learning with Big Data on CPUs, GPUs, in Parallel, and on the Cloud
		Deep Learning Using Simulink
		Deep Learning Interpretability
		Deep Learning Customization
		Deep Learning Import and Export
	Pretrained Deep Neural Networks
		Compare Pretrained Networks
		Load Pretrained Networks
		Visualize Pretrained Networks
		Feature Extraction
		Transfer Learning
		Import and Export Networks
		Pretrained Networks for Audio Applications
		Pretrained Models on GitHub
	Learn About Convolutional Neural Networks
	Example Deep Learning Networks Architectures
	Multiple-Input and Multiple-Output Networks
		Multiple-Input Networks
		Multiple-Output Networks
	List of Deep Learning Layers
		Deep Learning Layers
	Specify Layers of Convolutional Neural Network
		Image Input Layer
		Convolutional Layer
		Batch Normalization Layer
		ReLU Layer
		Cross Channel Normalization (Local Response Normalization) Layer
		Max and Average Pooling Layers
		Dropout Layer
		Fully Connected Layer
		Output Layers
	Set Up Parameters and Train Convolutional Neural Network
		Specify Solver and Maximum Number of Epochs
		Specify and Modify Learning Rate
		Specify Validation Data
		Select Hardware Resource
		Save Checkpoint Networks and Resume Training
		Set Up Parameters in Convolutional and Fully Connected Layers
		Train Your Network
	Train Network with Numeric Features
	Train Network on Image and Feature Data
	Compare Activation Layers
	Deep Learning Tips and Tricks
		Choose Network Architecture
		Choose Training Options
		Improve Training Accuracy
		Fix Errors in Training
		Prepare and Preprocess Data
		Use Available Hardware
		Fix Errors With Loading from MAT-Files
	Long Short-Term Memory Networks
		LSTM Network Architecture
		Layers
		Classification, Prediction, and Forecasting
		Sequence Padding, Truncation, and Splitting
		Normalize Sequence Data
		Out-of-Memory Data
		Visualization
		LSTM Layer Architecture
Deep Network Designer
	Transfer Learning with Deep Network Designer
	Build Networks with Deep Network Designer
		Transfer Learning
		Image Classification
		Sequence Classification
		Numeric Data Classification
		Convert Classification Network into Regression Network
		Multiple-Input and Multiple-Output Networks
		Deep Networks
		Advanced Deep Learning Applications
		dlnetwork for Custom Training Loops
		Check Network
	Train Networks Using Deep Network Designer
		Select Training Options
		Train Network
		Next Steps
	Import Custom Layer into Deep Network Designer
	Import Data into Deep Network Designer
		Import Data
		Image Augmentation
		Validation Data
	Create Simple Sequence Classification Network Using Deep Network Designer
	Train Network for Time Series Forecasting Using Deep Network Designer
	Generate MATLAB Code from Deep Network Designer
		Generate MATLAB Code to Recreate Network Layers
		Generate MATLAB Code to Train Network
	Image-to-Image Regression in Deep Network Designer
	Generate Experiment Using Deep Network Designer
	Transfer Learning with Pretrained Audio Networks in Deep Network Designer
	Export Image Classification Network from Deep Network Designer to Simulink
Deep Learning with Images
	Classify Webcam Images Using Deep Learning
	Train Deep Learning Network to Classify New Images
	Train Residual Network for Image Classification
	Classify Image Using GoogLeNet
	Extract Image Features Using Pretrained Network
	Transfer Learning Using Pretrained Network
	Transfer Learning Using AlexNet
	Create Simple Deep Learning Network for Classification
	Train Convolutional Neural Network for Regression
	Train Network with Multiple Outputs
	Convert Classification Network into Regression Network
	Train Generative Adversarial Network (GAN)
	Train Conditional Generative Adversarial Network (CGAN)
	Train Wasserstein GAN with Gradient Penalty (WGAN-GP)
	Train Fast Style Transfer Network
	Train a Siamese Network to Compare Images
	Train a Siamese Network for Dimensionality Reduction
	Train Neural ODE Network
	Train Variational Autoencoder (VAE) to Generate Images
	Lane and Vehicle Detection in Simulink Using Deep Learning
	Classify ECG Signals in Simulink Using Deep Learning
	Classify Images in Simulink Using GoogLeNet
	Multilabel Image Classification Using Deep Learning
	Acceleration for Simulink Deep Learning Models
		Run Acceleration Mode from the User Interface
		Run Acceleration Mode Programmatically
Deep Learning with Time Series, Sequences, and Text
	Sequence Classification Using Deep Learning
	Sequence Classification Using 1-D Convolutions
	Time Series Forecasting Using Deep Learning
	Train Speech Command Recognition Model Using Deep Learning
	Sequence-to-Sequence Classification Using Deep Learning
	Sequence-to-Sequence Regression Using Deep Learning
	Sequence-to-One Regression Using Deep Learning
	Train Network with Complex-Valued Data
	Train Network with LSTM Projected Layer
	Predict Battery State of Charge Using Deep Learning
	Classify Videos Using Deep Learning
	Classify Videos Using Deep Learning with Custom Training Loop
	Train Sequence Classification Network Using Data With Imbalanced Classes
	Sequence-to-Sequence Classification Using 1-D Convolutions
	Time Series Anomaly Detection Using Deep Learning
	Sequence Classification Using CNN-LSTM Network
	Train Latent ODE Network with Irregularly Sampled Time-Series Data
	Multivariate Time Series Anomaly Detection Using Graph Neural Network
	Classify Text Data Using Deep Learning
	Classify Text Data Using Convolutional Neural Network
	Multilabel Text Classification Using Deep Learning
	Classify Text Data Using Custom Training Loop
	Generate Text Using Autoencoders
	Define Text Encoder Model Function
	Define Text Decoder Model Function
	Sequence-to-Sequence Translation Using Attention
	Generate Text Using Deep Learning
	Pride and Prejudice and MATLAB
	Word-By-Word Text Generation Using Deep Learning
	Image Captioning Using Attention
	Language Translation Using Deep Learning
	Predict and Update Network State in Simulink
	Classify and Update Network State in Simulink
	Time Series Prediction in Simulink Using Deep Learning Network
	Battery State of Charge Estimation in Simulink Using Deep Learning Network
	Improve Performance of Deep Learning Simulations in Simulink
	Physical System Modeling Using LSTM Network in Simulink
Deep Learning Tuning and Visualization
	Explore Network Predictions Using Deep Learning Visualization Techniques
	Deep Dream Images Using GoogLeNet
	Grad-CAM Reveals the Why Behind Deep Learning Decisions
	Interpret Deep Learning Time-Series Classifications Using Grad-CAM
	Understand Network Predictions Using Occlusion
	Investigate Classification Decisions Using Gradient Attribution Techniques
	Understand Network Predictions Using LIME
	Investigate Spectrogram Classifications Using LIME
	Interpret Deep Network Predictions on Tabular Data Using LIME
	Explore Semantic Segmentation Network Using Grad-CAM
	Investigate Audio Classifications Using Deep Learning Interpretability Techniques
	Generate Untargeted and Targeted Adversarial Examples for Image Classification
	Train Image Classification Network Robust to Adversarial Examples
	Generate Adversarial Examples for Semantic Segmentation
	Resume Training from Checkpoint Network
	Deep Learning Using Bayesian Optimization
	Train Deep Learning Networks in Parallel
	Monitor Deep Learning Training Progress
	Customize Output During Deep Learning Network Training
	Detect Vanishing Gradients in Deep Neural Networks by Plotting Gradient Distributions
	Investigate Network Predictions Using Class Activation Mapping
	View Network Behavior Using tsne
	Visualize Activations of a Convolutional Neural Network
	Visualize Activations of LSTM Network
	Visualize Features of a Convolutional Neural Network
	Visualize Image Classifications Using Maximal and Minimal Activating Images
	Monitor GAN Training Progress and Identify Common Failure Modes
		Convergence Failure
		Mode Collapse
	Deep Learning Visualization Methods
		Visualization Methods
		Interpretability Methods for Nonimage Data
	ROC Curve and Performance Metrics
		Introduction to ROC Curve
		Performance Curve with MATLAB
		ROC Curve for Multiclass Classification
		Performance Metrics
		Classification Scores and Thresholds
		Pointwise Confidence Intervals
	Compare Deep Learning Models Using ROC Curves
Manage Deep Learning Experiments
	Create a Deep Learning Experiment for Classification
	Create a Deep Learning Experiment for Regression
	Use Experiment Manager to Train Networks in Parallel
		Set Up Parallel Environment
	Evaluate Deep Learning Experiments by Using Metric Functions
	Tune Experiment Hyperparameters by Using Bayesian Optimization
	Try Multiple Pretrained Networks for Transfer Learning
	Experiment with Weight Initializers for Transfer Learning
	Choose Training Configurations for LSTM Using Bayesian Optimization
	Run a Custom Training Experiment for Image Comparison
	Use Experiment Manager to Train Generative Adversarial Networks (GANs)
	Use Bayesian Optimization in Custom Training Experiments
	Custom Training with Multiple GPUs in Experiment Manager
	Offload Experiments as Batch Jobs to Cluster
		Create Batch Job on Cluster
		Track Progress of Batch Job
		Interrupt Training in Batch Job
		Retrieve Results and Clean Up Data
	Keyboard Shortcuts for Experiment Manager
		Shortcuts for General Navigation
		Shortcuts for Experiment Browser
		Shortcuts for Results Table
	Debug Experiments for Deep Learning
		Debug Built-In Training Experiments
		Debug Custom Training Experiments
Deep Learning in Parallel and the Cloud
	Scale Up Deep Learning in Parallel, on GPUs, and in the Cloud
		Train Single Network in Parallel
		Train Multiple Networks in Parallel
		Batch Deep Learning
		Manage Cluster Profiles and Automatic Pool Creation
		Deep Learning Precision
	Deep Learning in the Cloud
		Access MATLAB in the Cloud
		Work with Big Data in the Cloud
	Deep Learning with MATLAB on Multiple GPUs
		Use Multiple GPUs in Local Machine
		Use Multiple GPUs in Cluster
		Optimize Mini-Batch Size and Learning Rate
		Select Particular GPUs to Use for Training
		Train Multiple Networks on Multiple GPUs
		Advanced Support for Fast Multi-Node GPU Communication
	Deep Learning with Big Data
		Work with Big Data in Parallel
		Preprocess Data in Background
		Work with Big Data in the Cloud
		Preprocess Data for Custom Training Loops
	Run Custom Training Loops on a GPU and in Parallel
		Train Network on GPU
		Train Single Network in Parallel
		Train Multiple Networks in Parallel
		Use Experiment Manager to Train in Parallel
	Cloud AI Workflow Using the Deep Learning Container
	Train Network in the Cloud Using Automatic Parallel Support
	Use parfeval to Train Multiple Deep Learning Networks
	Send Deep Learning Batch Job to Cluster
	Train Network Using Automatic Multi-GPU Support
	Use parfor to Train Multiple Deep Learning Networks
	Upload Deep Learning Data to the Cloud
	Train Network in Parallel with Custom Training Loop
	Train Network Using Federated Learning
	Train Network on Amazon Web Services Using MATLAB Deep Learning Container
	Use Amazon S3 Buckets with MATLAB Deep Learning Container
	Use Experiment Manager in the Cloud with MATLAB Deep Learning Container
Computer Vision Examples
	Gesture Recognition using Videos and Deep Learning
	Code Generation for Object Detection by Using Single Shot Multibox Detector
	Point Cloud Classification Using PointNet Deep Learning
	Activity Recognition from Video and Optical Flow Data Using Deep Learning
	Import Pretrained ONNX YOLO v2 Object Detector
	Export YOLO v2 Object Detector to ONNX
	Object Detection Using SSD Deep Learning
	Object Detection Using YOLO v3 Deep Learning
	Object Detection Using YOLO v4 Deep Learning
	Object Detection Using YOLO v2 Deep Learning
	Semantic Segmentation Using Deep Learning
	Semantic Segmentation Using Dilated Convolutions
	Train Simple Semantic Segmentation Network in Deep Network Designer
	Semantic Segmentation of Multispectral Images Using Deep Learning
	3-D Brain Tumor Segmentation Using Deep Learning
	Define Custom Pixel Classification Layer with Tversky Loss
	Train Object Detector Using R-CNN Deep Learning
	Object Detection Using Faster R-CNN Deep Learning
	Perform Instance Segmentation Using Mask R-CNN
	Estimate Body Pose Using Deep Learning
	Generate Image from Segmentation Map Using Deep Learning
Image Processing Examples
	Remove Noise from Color Image Using Pretrained Neural Network
	Increase Image Resolution Using Deep Learning
	JPEG Image Deblocking Using Deep Learning
	Image Processing Operator Approximation Using Deep Learning
	Develop Camera Processing Pipeline Using Deep Learning
	Brighten Extremely Dark Images Using Deep Learning
	Classify Tumors in Multiresolution Blocked Images
	Unsupervised Day-to-Dusk Image Translation Using UNIT
	Quantify Image Quality Using Neural Image Assessment
	Neural Style Transfer Using Deep Learning
	Unsupervised Medical Image Denoising Using CycleGAN
	Unsupervised Medical Image Denoising Using UNIT
	Detect Image Anomalies Using Explainable One-Class Classification Neural Network
	Classify Defects on Wafer Maps Using Deep Learning
	Detect Image Anomalies Using Pretrained ResNet-18 Feature Embeddings
	Segment Lungs from CT Scan Using Pretrained Neural Network
	Brain MRI Segmentation Using Pretrained 3-D U-Net Network
	Breast Tumor Segmentation from Ultrasound Using Deep Learning
Automated Driving Examples
	Train a Deep Learning Vehicle Detector
	Create Occupancy Grid Using Monocular Camera and Semantic Segmentation
	Train Deep Learning Semantic Segmentation Network Using 3-D Simulation Data
Lidar Examples
	Code Generation for Lidar Object Detection Using SqueezeSegV2 Network
	Lidar Object Detection Using Complex-YOLO v4 Network
	Aerial Lidar Semantic Segmentation Using PointNet++ Deep Learning
	Code Generation For Aerial Lidar Semantic Segmentation Using PointNet++ Deep Learning
	Lidar Point Cloud Semantic Segmentation Using PointSeg Deep Learning Network
	Lidar Point Cloud Semantic Segmentation Using SqueezeSegV2 Deep Learning Network
	Code Generation for Lidar Point Cloud Segmentation Network
	Lidar 3-D Object Detection Using PointPillars Deep Learning
Signal Processing Examples
	Learn Pre-Emphasis Filter Using Deep Learning
	Hand Gesture Classification Using Radar Signals and Deep Learning
	Waveform Segmentation Using Deep Learning
	Classify ECG Signals Using Long Short-Term Memory Networks
	Generate Synthetic Signals Using Conditional GAN
	Classify Time Series Using Wavelet Analysis and Deep Learning
	Deploy Signal Classifier on NVIDIA Jetson Using Wavelet Analysis and Deep Learning
	Deploy Signal Classifier Using Wavelets and Deep Learning on Raspberry Pi
	Deploy Signal Segmentation Deep Network on Raspberry Pi
	Anomaly Detection Using Autoencoder and Wavelets
	Fault Detection Using Wavelet Scattering and Recurrent Deep Networks
	Parasite Classification Using Wavelet Scattering and Deep Learning
	Detect Anomalies Using Wavelet Scattering with Autoencoders
	Denoise Signals with Adversarial Learning Denoiser Model
	Human Health Monitoring Using Continuous Wave Radar and Deep Learning
Wireless Comm Examples
	Train DQN Agent for Beam Selection
	CSI Feedback with Autoencoders
	Modulation Classification by Using FPGA
	Neural Network for Digital Predistortion Design - Offline Training
	Neural Network for Beam Selection
	Spectrum Sensing with Deep Learning to Identify 5G and LTE Signals
	Autoencoders for Wireless Communications
	Modulation Classification with Deep Learning
	Training and Testing a Neural Network for LLR Estimation
	Design a Deep Neural Network with Simulated Data to Detect WLAN Router Impersonation
	Test a Deep Neural Network with Captured Data to Detect WLAN Router Impersonation
Audio Examples
	Transfer Learning with Pretrained Audio Networks
	Speech Command Recognition in Simulink
	Speaker Identification Using Custom SincNet Layer and Deep Learning
	Dereverberate Speech Using Deep Learning Networks
	Speaker Recognition Using x-vectors
	Speaker Diarization Using x-vectors
	Train Spoken Digit Recognition Network Using Out-of-Memory Audio Data
	Train Spoken Digit Recognition Network Using Out-of-Memory Features
	Keyword Spotting in Noise Code Generation with Intel MKL-DNN
	Keyword Spotting in Noise Code Generation on Raspberry Pi
	Speech Command Recognition Code Generation on Raspberry Pi
	Speech Command Recognition Code Generation with Intel MKL-DNN
	Train Generative Adversarial Network (GAN) for Sound Synthesis
	Sequential Feature Selection for Audio Features
	Acoustic Scene Recognition Using Late Fusion
	Keyword Spotting in Noise Using MFCC and LSTM Networks
	Speech Emotion Recognition
	Spoken Digit Recognition with Wavelet Scattering and Deep Learning
	Cocktail Party Source Separation Using Deep Learning Networks
	Voice Activity Detection in Noise Using Deep Learning
	Denoise Speech Using Deep Learning Networks
	Accelerate Audio Deep Learning Using GPU-Based Feature Extraction
	Acoustics-Based Machine Fault Recognition
	Acoustics-Based Machine Fault Recognition Code Generation with Intel MKL-DNN
	Acoustics-Based Machine Fault Recognition Code Generation on Raspberry Pi
	End-to-End Deep Speech Separation
	Train 3-D Sound Event Localization and Detection (SELD) Using Deep Learning
	3-D Sound Event Localization and Detection Using Trained Recurrent Convolutional Neural Network
	Speech Command Recognition Code Generation with Intel MKL-DNN Using Simulink
	Speech Command Recognition on Raspberry Pi Using Simulink
	Audio-Based Anomaly Detection for Machine Health Monitoring
	3-D Speech Enhancement Using Trained Filter and Sum Network
	Train 3-D Speech Enhancement Network Using Deep Learning
	Audio Transfer Learning Using Experiment Manager
Reinforcement Learning Examples
	Reinforcement Learning Using Deep Neural Networks
		Reinforcement Learning Workflow
		Reinforcement Learning Environments
		Reinforcement Learning Agents
		Create Deep Neural Network Policies and Value Functions
		Train Reinforcement Learning Agents
		Deploy Trained Policies
	Create Simulink Environment and Train Agent
	Train DDPG Agent to Swing Up and Balance Pendulum with Image Observation
	Create Agent Using Deep Network Designer and Train Using Image Observations
	Imitate MPC Controller for Lane Keeping Assist
	Train DDPG Agent to Control Flying Robot
	Train Biped Robot to Walk Using Reinforcement Learning Agents
	Train Humanoid Walker
	Train DDPG Agent for Adaptive Cruise Control
	Train DQN Agent for Lane Keeping Assist Using Parallel Computing
	Train DDPG Agent for Path-Following Control
	Train PPO Agent for Automatic Parking Valet
Predictive Maintenance Examples
	Chemical Process Fault Detection Using Deep Learning
	Rolling Element Bearing Fault Diagnosis Using Deep Learning
	Remaining Useful Life Estimation Using Convolutional Neural Network
	Anomaly Detection in Industrial Machinery Using Three-Axis Vibration Data
	Battery Cycle Life Prediction Using Deep Learning
Computational Finance Examples
	Compare Deep Learning Networks for Credit Default Prediction
	Interpret and Stress-Test Deep Learning Networks for Probability of Default
	Hedge Options Using Reinforcement Learning Toolbox™
	Use Deep Learning to Approximate Barrier Option Prices with Heston Model
	Backtest Strategies Using Deep Learning
Import, Export, and Customization
	Train Deep Learning Model in MATLAB
		Training Methods
		Decisions
	Define Custom Deep Learning Layers
		Layer Templates
		Intermediate Layer Architecture
		Output Layer Architecture
		Check Validity of Custom Layer
	Define Custom Deep Learning Intermediate Layers
		Intermediate Layer Architecture
		Intermediate Layer Template
		Formatted Inputs and Outputs
		Custom Layer Acceleration
		Intermediate Layer Properties
		Forward Functions
		Reset State Function
		Backward Function
		GPU Compatibility
		Check Validity of Layer
	Define Custom Deep Learning Output Layers
		Output Layer Architecture
		Output Layer Templates
		Custom Layer Acceleration
		Output Layer Properties
		Forward Loss Function
		Backward Loss Function
		GPU Compatibility
		Check Validity of Layer
	Define Custom Deep Learning Layer with Learnable Parameters
		Intermediate Layer Template
		Name Layer and Specify Superclasses
		Declare Properties and Learnable Parameters
		Create Constructor Function
		Create Initialize Function
		Create Forward Functions
		Completed Layer
		GPU Compatibility
		Check Validity of Custom Layer Using checkLayer
		Include Custom Layer in Network
	Define Custom Deep Learning Layer with Multiple Inputs
		Intermediate Layer Template
		Name Layer and Specify Superclasses
		Declare Properties and Learnable Parameters
		Create Constructor Function
		Create Forward Functions
		Completed Layer
		GPU Compatibility
		Check Validity of Layer with Multiple Inputs
		Use Custom Weighted Addition Layer in Network
	Define Custom Deep Learning Layer with Formatted Inputs
		Intermediate Layer Template
		Name Layer and Specify Superclasses
		Declare Properties and Learnable Parameters
		Create Constructor Function
		Create Initialize Function
		Create Forward Functions
		Completed Layer
		GPU Compatibility
		Include Custom Layer in Network
	Define Custom Recurrent Deep Learning Layer
		Intermediate Layer Template
		Name Layer
		Declare Properties, State, and Learnable Parameters
		Create Constructor Function
		Create Initialize Function
		Create Predict Function
		Create Reset State Function
		Completed Layer
		GPU Compatibility
		Include Custom Layer in Network
	Define Custom Classification Output Layer
		Classification Output Layer Template
		Name the Layer and Specify Superclasses
		Declare Layer Properties
		Create Constructor Function
		Create Forward Loss Function
		Completed Layer
		GPU Compatibility
		Check Output Layer Validity
		Include Custom Classification Output Layer in Network
	Define Custom Regression Output Layer
		Regression Output Layer Template
		Name the Layer and Specify Superclasses
		Declare Layer Properties
		Create Constructor Function
		Create Forward Loss Function
		Completed Layer
		GPU Compatibility
		Check Output Layer Validity
		Include Custom Regression Output Layer in Network
	Specify Custom Layer Backward Function
		Create Custom Layer
		Create Backward Function
		Complete Layer
		GPU Compatibility
	Specify Custom Output Layer Backward Loss Function
		Create Custom Layer
		Create Backward Loss Function
		Complete Layer
		GPU Compatibility
	Custom Layer Function Acceleration
		Acceleration Considerations
	Deep Learning Network Composition
		Automatically Initialize Learnable dlnetwork Objects for Training
		Predict and Forward Functions
		GPU Compatibility
	Define Nested Deep Learning Layer
		Intermediate Layer Template
		Name Layer and Specify Superclasses
		Declare Properties and Learnable Parameters
		Create Constructor Function
		Create Forward Functions
		Completed Layer
		GPU Compatibility
	Train Deep Learning Network with Nested Layers
	Define Custom Deep Learning Layer for Code Generation
		Intermediate Layer Template
		Name Layer and Specify Superclasses
		Specify Code Generation Pragma
		Declare Properties and Learnable Parameters
		Create Constructor Function
		Create Forward Functions
		Completed Layer
		Check Custom Layer for Code Generation Compatibility
	Check Custom Layer Validity
		Check Custom Layer Validity
		List of Tests
		Generated Data
		Diagnostics
	Specify Custom Weight Initialization Function
	Compare Layer Weight Initializers
	Assemble Network from Pretrained Keras Layers
	Replace Unsupported Keras Layer with Function Layer
	Assemble Multiple-Output Network for Prediction
	Automatic Differentiation Background
		What Is Automatic Differentiation?
		Forward Mode
		Reverse Mode
	Use Automatic Differentiation In Deep Learning Toolbox
		Custom Training and Calculations Using Automatic Differentiation
		Use dlgradient and dlfeval Together for Automatic Differentiation
		Derivative Trace
		Characteristics of Automatic Derivatives
	Define Custom Training Loops, Loss Functions, and Networks
		Define Deep Learning Network for Custom Training Loops
		Specify Loss Functions
		Update Learnable Parameters Using Automatic Differentiation
	Specify Training Options in Custom Training Loop
		Solver Options
		Learn Rate
		Plots
		Verbose Output
		Mini-Batch Size
		Number of Epochs
		Validation
		L2 Regularization
		Gradient Clipping
		Single CPU or GPU Training
		Checkpoints
	Train Network Using Custom Training Loop
	Define Model Loss Function for Custom Training Loop
		Create Model Loss Function for Model Defined as dlnetwork Object
		Create Model Loss Function for Model Defined as Function
		Evaluate Model Loss Function
		Update Learnable Parameters Using Gradients
		Use Model Loss Function in Custom Training Loop
		Debug Model Loss Functions
	Update Batch Normalization Statistics in Custom Training Loop
	Train Robust Deep Learning Network with Jacobian Regularization
	Make Predictions Using dlnetwork Object
	Train Network Using Model Function
	Update Batch Normalization Statistics Using Model Function
	Make Predictions Using Model Function
	Initialize Learnable Parameters for Model Function
		Default Layer Initializations
		Learnable Parameter Sizes
		Glorot Initialization
		He Initialization
		Gaussian Initialization
		Uniform Initialization
		Orthogonal Initialization
		Unit Forget Gate Initialization
		Ones Initialization
		Zeros Initialization
		Storing Learnable Parameters
	Deep Learning Function Acceleration for Custom Training Loops
		Accelerate Deep Learning Function Directly
		Accelerate Parts of Deep Learning Function
		Reusing Caches
		Storing and Clearing Caches
		Acceleration Considerations
	Accelerate Custom Training Loop Functions
	Evaluate Performance of Accelerated Deep Learning Function
	Check Accelerated Deep Learning Function Outputs
	Solve Partial Differential Equations Using Deep Learning
	Solve Partial Differential Equation with LBFGS Method and Deep Learning
	Solve Ordinary Differential Equation Using Neural Network
	Dynamical System Modeling Using Neural ODE
	Node Classification Using Graph Convolutional Network
	Multilabel Graph Classification Using Graph Attention Networks
	Train Network Using Cyclical Learning Rate for Snapshot Ensembling
	Interoperability Between Deep Learning Toolbox, TensorFlow, PyTorch, and ONNX
		Support Packages for Interoperability
		Functions that Import Deep Learning Networks
		Visualize Imported Network
		Predict with Imported Model
		Transfer Learning with Imported Network
		Deploy Imported Network
		Functions that Export Networks and Layer Graphs
	Tips on Importing Models from TensorFlow, PyTorch, and ONNX
		Import Functions of Deep Learning Toolbox
		Recommended Functions to Import TensorFlow Models
		Autogenerated Custom Layers
		Placeholder Layers
		Input Dimension Ordering
		Data Formats for Prediction with dlnetwork
		Input Data Preprocessing
	Deploy Imported TensorFlow Model with MATLAB Compiler
	Select Function to Import ONNX Pretrained Network
		Decisions
		Actions
	Classify Images in Simulink with Imported TensorFlow Network
	Inference Comparison Between TensorFlow and Imported Networks for Image Classification
	Inference Comparison Between ONNX and Imported Networks for Image Classification
	List of Functions with dlarray Support
		Deep Learning Toolbox Functions with dlarray Support
		Domain-Specific Functions with dlarray Support
		MATLAB Functions with dlarray Support
		Notable dlarray Behaviors
	Monitor Custom Training Loop Progress
		Create Training Progress Monitor
		Training Progress Window
		Monitor Custom Training Loop Progress During Training
	Train Bayesian Neural Network
Deep Learning Data Preprocessing
	Datastores for Deep Learning
		Select Datastore
		Input Datastore for Training, Validation, and Inference
		Specify Read Size and Mini-Batch Size
		Transform and Combine Datastores
		Use Datastore for Parallel Training and Background Dispatching
	Create and Explore Datastore for Image Classification
	Preprocess Images for Deep Learning
		Resize Images Using Rescaling and Cropping
		Augment Images for Training with Random Geometric Transformations
		Perform Additional Image Processing Operations Using Built-In Datastores
		Apply Custom Image Processing Pipelines Using Combine and Transform
	Preprocess Volumes for Deep Learning
		Read Volumetric Data
		Pair Image and Label Data
		Preprocess Volumetric Data
		Examples
	Preprocess Data for Domain-Specific Deep Learning Applications
		Image Processing Applications
		Object Detection
		Semantic Segmentation
		Lidar Processing Applications
		Signal Processing Applications
		Audio Processing Applications
		Text Analytics
	Develop Custom Mini-Batch Datastore
		Overview
		Implement MiniBatchable Datastore
		Add Support for Shuffling
		Validate Custom Mini-Batch Datastore
	Augment Images for Deep Learning Workflows
	Augment Pixel Labels for Semantic Segmentation
	Augment Bounding Boxes for Object Detection
	Prepare Datastore for Image-to-Image Regression
	Train Network Using Out-of-Memory Sequence Data
	Train Network Using Custom Mini-Batch Datastore for Sequence Data
	Classify Out-of-Memory Text Data Using Deep Learning
	Classify Out-of-Memory Text Data Using Custom Mini-Batch Datastore
	Data Sets for Deep Learning
		Image Data Sets
		Time Series and Signal Data Sets
		Video Data Sets
		Text Data Sets
		Audio Data Sets
		Point Cloud Data Sets
	Choose an App to Label Ground Truth Data
Deep Learning Code Generation
	Code Generation for Deep Learning Networks
	Code Generation for Semantic Segmentation Network
	Lane Detection Optimized with GPU Coder
	Code Generation for a Sequence-to-Sequence LSTM Network
	Deep Learning Prediction on ARM Mali GPU
	Code Generation for Object Detection by Using YOLO v2
	Code Generation For Object Detection Using YOLO v3 Deep Learning
	Code Generation for Object Detection Using YOLO v4 Deep Learning
	Deep Learning Prediction with NVIDIA TensorRT Library
	Traffic Sign Detection and Recognition
	Logo Recognition Network
	Code Generation for Denoising Deep Neural Network
	Train and Deploy Fully Convolutional Networks for Semantic Segmentation
	Code Generation for Semantic Segmentation Network That Uses U-net
	Code Generation for Deep Learning on ARM Targets
	Deep Learning Prediction with ARM Compute Using codegen
	Deep Learning Code Generation on Intel Targets for Different Batch Sizes
	Generate C++ Code for Object Detection Using YOLO v2 and Intel MKL-DNN
	Code Generation and Deployment of MobileNet-v2 Network to Raspberry Pi
	Code Generation for Semantic Segmentation Application on Intel CPUs That Uses U-Net
	Code Generation for Semantic Segmentation Application on ARM Neon Targets That Uses U-Net
	Code Generation for LSTM Network on Raspberry Pi
	Code Generation for LSTM Network That Uses Intel MKL-DNN
	Cross Compile Deep Learning Code for ARM Neon Targets
	Generate Generic C/C++ Code for Sequence-to-Sequence Regression That Uses Deep Learning
	Quantize Residual Network Trained for Image Classification and Generate CUDA Code
	Quantize Layers in Object Detectors and Generate CUDA Code
	Parameter Pruning and Quantization of Image Classification Network
	Prune Image Classification Network Using Taylor Scores
	Quantization Workflow Prerequisites
		Prerequisites for All Quantization Workflows
		Supported Networks and Layers
		Prerequisites for Calibration
		Prerequisites for Quantization
		Prerequisites for Validation
	Quantization of Deep Neural Networks
		Precision and Range
		Histograms of Dynamic Ranges
	Prune Filters in a Detection Network Using Taylor Scores
	Prerequisites for Deep Learning with TensorFlow Lite Models
		MathWorks Products
		Third-Party Hardware and Software
		Environment Variables
	Generate Code for TensorFlow Lite (TFLite) Model and Deploy on Raspberry Pi
	Deploy Super Resolution Application That Uses TensorFlow Lite (TFLite) Model on Host and Raspberry Pi
Neural Network Objects, Data, and Training Styles
	Workflow for Neural Network Design
	Four Levels of Neural Network Design
	Neuron Model
		Simple Neuron
		Transfer Functions
		Neuron with Vector Input
	Neural Network Architectures
		One Layer of Neurons
		Multiple Layers of Neurons
		Input and Output Processing Functions
	Create Neural Network Object
	Configure Shallow Neural Network Inputs and Outputs
	Understanding Shallow Network Data Structures
		Simulation with Concurrent Inputs in a Static Network
		Simulation with Sequential Inputs in a Dynamic Network
		Simulation with Concurrent Inputs in a Dynamic Network
	Neural Network Training Concepts
		Incremental Training with adapt
		Batch Training
		Training Feedback
Multilayer Shallow Neural Networks and Backpropagation Training
	Multilayer Shallow Neural Networks and Backpropagation Training
	Multilayer Shallow Neural Network Architecture
		Neuron Model (logsig, tansig, purelin)
		Feedforward Neural Network
	Prepare Data for Multilayer Shallow Neural Networks
	Choose Neural Network Input-Output Processing Functions
		Representing Unknown or Don't-Care Targets
	Divide Data for Optimal Neural Network Training
	Create, Configure, and Initialize Multilayer Shallow Neural Networks
		Other Related Architectures
		Initializing Weights (init)
	Train and Apply Multilayer Shallow Neural Networks
		Training Algorithms
		Training Example
		Use the Network
	Analyze Shallow Neural Network Performance After Training
		Improving Results
	Limitations and Cautions
Dynamic Neural Networks
	Introduction to Dynamic Neural Networks
	How Dynamic Neural Networks Work
		Feedforward and Recurrent Neural Networks
		Applications of Dynamic Networks
		Dynamic Network Structures
		Dynamic Network Training
	Design Time Series Time-Delay Neural Networks
		Prepare Input and Layer Delay States
	Design Time Series Distributed Delay Neural Networks
	Design Time Series NARX Feedback Neural Networks
		Multiple External Variables
	Design Layer-Recurrent Neural Networks
	Create Reference Model Controller with MATLAB Script
	Multiple Sequences with Dynamic Neural Networks
	Neural Network Time-Series Utilities
	Train Neural Networks with Error Weights
	Normalize Errors of Multiple Outputs
	Multistep Neural Network Prediction
		Set Up in Open-Loop Mode
		Multistep Closed-Loop Prediction From Initial Conditions
		Multistep Closed-Loop Prediction Following Known Sequence
		Following Closed-Loop Simulation with Open-Loop Simulation
Control Systems
	Introduction to Neural Network Control Systems
	Design Neural Network Predictive Controller in Simulink
		System Identification
		Predictive Control
		Use the Neural Network Predictive Controller Block
	Design NARMA-L2 Neural Controller in Simulink
		Identification of the NARMA-L2 Model
		NARMA-L2 Controller
		Use the NARMA-L2 Controller Block
	Design Model-Reference Neural Controller in Simulink
		Use the Model Reference Controller Block
	Import-Export Neural Network Simulink Control Systems
		Import and Export Networks
		Import and Export Training Data
Radial Basis Neural Networks
	Introduction to Radial Basis Neural Networks
		Important Radial Basis Functions
	Radial Basis Neural Networks
		Neuron Model
		Network Architecture
		Exact Design (newrbe)
		More Efficient Design (newrb)
		Examples
	Probabilistic Neural Networks
		Network Architecture
		Design (newpnn)
	Generalized Regression Neural Networks
		Network Architecture
		Design (newgrnn)
Self-Organizing and Learning Vector Quantization Networks
	Introduction to Self-Organizing and LVQ
		Important Self-Organizing and LVQ Functions
	Cluster with a Competitive Neural Network
		Architecture
		Create a Competitive Neural Network
		Kohonen Learning Rule (learnk)
		Bias Learning Rule (learncon)
		Training
		Graphical Example
	Cluster with Self-Organizing Map Neural Network
		Topologies (gridtop, hextop, randtop)
		Distance Functions (dist, linkdist, mandist, boxdist)
		Architecture
		Create a Self-Organizing Map Neural Network (selforgmap)
		Training (learnsomb)
		Examples
	Learning Vector Quantization (LVQ) Neural Networks
		Architecture
		Creating an LVQ Network
		LVQ1 Learning Rule (learnlv1)
		Training
		Supplemental LVQ2.1 Learning Rule (learnlv2)
Adaptive Filters and Adaptive Training
	Adaptive Neural Network Filters
		Adaptive Functions
		Linear Neuron Model
		Adaptive Linear Network Architecture
		Least Mean Square Error
		LMS Algorithm (learnwh)
		Adaptive Filtering (adapt)
Advanced Topics
	Shallow Neural Networks with Parallel and GPU Computing
		Modes of Parallelism
		Distributed Computing
		Single GPU Computing
		Distributed GPU Computing
		Parallel Time Series
		Parallel Availability, Fallbacks, and Feedback
	Optimize Neural Network Training Speed and Memory
		Memory Reduction
		Fast Elliot Sigmoid
	Choose a Multilayer Neural Network Training Function
		SIN Data Set
		PARITY Data Set
		ENGINE Data Set
		CANCER Data Set
		CHOLESTEROL Data Set
		DIABETES Data Set
		Summary
	Improve Shallow Neural Network Generalization and Avoid Overfitting
		Retraining Neural Networks
		Multiple Neural Networks
		Early Stopping
		Index Data Division (divideind)
		Random Data Division (dividerand)
		Block Data Division (divideblock)
		Interleaved Data Division (divideint)
		Regularization
		Summary and Discussion of Early Stopping and Regularization
		Posttraining Analysis (regression)
	Edit Shallow Neural Network Properties
		Custom Network
		Network Definition
		Network Behavior
	Custom Neural Network Helper Functions
	Automatically Save Checkpoints During Neural Network Training
	Deploy Shallow Neural Network Functions
		Deployment Functions and Tools for Trained Networks
		Generate Neural Network Functions for Application Deployment
		Generate Simulink Diagrams
	Deploy Training of Shallow Neural Networks
Historical Neural Networks
	Historical Neural Networks Overview
	Perceptron Neural Networks
		Neuron Model
		Perceptron Architecture
		Create a Perceptron
		Perceptron Learning Rule (learnp)
		Training (train)
		Limitations and Cautions
	Linear Neural Networks
		Neuron Model
		Network Architecture
		Least Mean Square Error
		Linear System Design (newlind)
		Linear Networks with Delays
		LMS Algorithm (learnwh)
		Linear Classification (train)
		Limitations and Cautions
Neural Network Object Reference
	Neural Network Object Properties
		General
		Architecture
		Subobject Structures
		Functions
		Weight and Bias Values
	Neural Network Subobject Properties
		Inputs
		Layers
		Outputs
		Biases
		Input Weights
		Layer Weights
Function Approximation, Clustering, and Control Examples
	Fit Data Using the Neural Net Fitting App
	Classify Patterns Using the Neural Net Pattern Recognition App
	Cluster Data Using the Neural Net Clustering App
	Fit Time Series Data Using the Neural Net Time Series App
	Body Fat Estimation
	Crab Classification
	Wine Classification
	Cancer Detection
	Character Recognition
	Train Stacked Autoencoders for Image Classification
	Iris Clustering
	Gene Expression Analysis
	Maglev Modeling
	Competitive Learning
	One-Dimensional Self-Organizing Map
	Two-Dimensional Self-Organizing Map
	Radial Basis Approximation
	Radial Basis Underlapping Neurons
	Radial Basis Overlapping Neurons
	GRNN Function Approximation
	PNN Classification
	Learning Vector Quantization
	Linear Prediction Design
	Adaptive Linear Prediction
	Classification with a Two-Input Perceptron
	Outlier Input Vectors
	Normalized Perceptron Rule
	Linearly Non-separable Vectors
	Pattern Association Showing Error Surface
	Training a Linear Neuron
	Linear Fit of Nonlinear Problem
	Underdetermined Problem
	Linearly Dependent Problem
	Too Large a Learning Rate
	Adaptive Noise Cancellation
Shallow Neural Networks Bibliography
	Shallow Neural Networks Bibliography
Mathematical Notation
	Mathematics and Code Equivalents
		Mathematics Notation to MATLAB Notation
		Figure Notation
Neural Network Blocks for the Simulink Environment
	Neural Network Simulink Block Library
		Transfer Function Blocks
		Net Input Blocks
		Weight Blocks
		Processing Blocks
	Deploy Shallow Neural Network Simulink Diagrams
		Example
		Suggested Exercises
		Generate Functions and Objects
Code Notes
	Deep Learning Toolbox Data Conventions
		Dimensions
		Variables